Imbalanced Data Classification Research Articles

Two-stage fine-tuning with ChatGPT data augmentation for learning class-imbalanced data

Classification of long-tailed distributed data is a challenging problem, which suffers from serious class imbalance and hence poor performance on tail classes, which have only a few samples. Owing to this paucity of samples, learning on the tail classes is especially challenging for fine-tuning when transferring a pretrained model to a downstream task. In this work, we present a simple modification of standard fine-tuning to cope with these challenges. Specifically, we propose a two-stage fine-tuning. In Stage 1, we fine-tune the final layer of the pretrained model with class-balanced augmented data, generated using ChatGPT. As a large generative language model, ChatGPT is capable of generating novel and contextually similar responses to a given prompt, which makes it an excellent candidate for data augmentation. In Stage 2, we perform the standard fine-tuning. Our modification has several benefits: (1) it leverages pretrained representations by only fine-tuning a small portion of the model parameters while keeping the rest untouched; (2) it allows the model to learn an initial representation of the specific task; and importantly (3) it protects the learning of tail classes from being at a disadvantage during the model updating. We conduct extensive experiments on synthetic datasets of both two-class and multi-class tasks of text classification as well as a real-world application to ADME (i.e., absorption, distribution, metabolism, and excretion) semantic drug labeling. The experimental results show that the proposed two-stage fine-tuning outperforms vanilla fine-tuning and state-of-the-art methods on the above datasets.

Correction: Phased progressive learning with coupling-regulation-imbalance loss for imbalanced data classification

Correction: Phased progressive learning with coupling-regulation-imbalance loss for imbalanced data classification

A MeanShift-guided oversampling with self-adaptive sizes for imbalanced data classification

The imbalanced data classification has gained popularity in machine learning research domain due to its prevalence in numerous applications and its difficulty. However, the majority of contemporary work primarily focuses on addressing between-class imbalance issues. Previous researches have shown that combined with other elements, such as within-class imbalance, small sample size and the presence of small disjuncts, the imbalanced data significantly increase the difficulties for the traditional classifiers to learn. Therefore, we propose a novel MeanShift-guided oversampling with self-adaptive sizes for imbalanced data classification. The proposed MeanShift-guided oversampling technique can simultaneously consider the distribution of minority class and majority class within the sphere with the current minority instance as its center, which can favor addressing small sample size and avoiding overlapping issues often caused by the nearest neighbor (NN)-based oversampling techniques. The incorporation of random vector and flexible cut-off mechanism for vector length can enhance the diversity among the generated synthetic minority instances and avoid overlapping, which makes it suitable for small sample size and small disjuncts problems. To address between-class and within-class imbalance issues, we also introduce a self-adaptive sizes assignment strategy for each minority instance to be oversampled, where the assigned size is inversely proportional to its density and its distance from the majority class. In addition to eliminating within-class imbalance, the strategy can ensure that the informative border minority instances have more opportunities to be oversampled, thus improving classification performance. Extensive experimental results on some datasets with different distributions and imbalance ratios show the proposed algorithm outperforms other compared ones with significant difference.

Performance of Random Oversampling, Random Undersampling, and SMOTE-NC Methods in Handling Imbalanced Class in Classification Models

One common challenge in classification modeling is the existence of imbalanced classes within the data. If the analysis continues with imbalanced classes, it is probable that the result will demonstrate inadequate performance when forecasting new data. Various approaches exist to rectify this class imbalance issue, such as random oversampling, random undersampling, and the Synthetic Minority Over-sampling Technique for Nominal and Continuous (SMOTE-NC). Each of these methods encompasses distinct techniques aimed at achieving balanced class distribution within the dataset. Comparison of classification performance on imbalanced classes handled by these three methods has never been carried out in previous research. Therefore, this study undertakes an evaluation of classification models (specifically Gradient Boosting, Random Forest, and Extremely Randomized Trees) in the context of imbalanced class data. The results of this research show that the random undersampling method used to balance the class distribution has the best performance on two classification models (Random Forest and Gradient Boosted Tree).

A Hybrid Approach for Binary Classification of Imbalanced Data

Binary classification with an imbalanced dataset is challenging. Models tend to consider all samples as belonging to the majority class. Although existing solutions such as sampling methods, cost-sensitive methods, and ensemble learning methods improve the poor accuracy of the minority class, these methods are limited by overfitting or cost parameters that are difficult to decide. This paper proposes a hybrid approach with dimension reduction that consists of data block construction, dimensionality reduction, and ensemble learning with deep neural network classifiers. The performance is evaluated on eight imbalanced public datasets in terms of recall, G-mean, AUC, F-measure, and balanced accuracy. The results show that the proposed model outperforms state-of-the-art methods.

Interpretable machine learning models for failure cause prediction in imbalanced oil pipeline data

Pipelines are critical arteries in the oil and gas industry and require massive capital investment to safely construct networks that transport hydrocarbons across diverse environments. However, these pipeline systems are prone to integrity failure, which results in significant economic losses and environmental damage. Accurate prediction of pipeline failure events using historical oil pipeline accident data enables asset managers to plan sufficient maintenance, rehabilitation, and repair activities to prevent catastrophic failures. However, learning the complex interdependencies between pipeline attributes and rare failure events presents several analytical challenges. This study proposes a novel machine learning (ML) framework to accurately predict pipeline failure causes on highly class-imbalanced data compiled by the United States Pipeline and Hazardous Materials Safety Administration. Natural language processing techniques were leveraged to extract informative features from unstructured text data. Furthermore, class imbalance in the dataset was addressed via oversampling and intrinsic cost-sensitive learning (CSL) strategies adapted for the multi-class case. Nine machine and deep learning architectures were benchmarked, with LightGBM demonstrating superior performance. The integration of CSL yielded an 86% F1 score and a 0.82 Cohen kappa score, significantly advancing prior research. This study leveraged a comprehensive Shapley Additive explanation analysis to interpret the predictions from the LightGBM algorithm, revealing the key factors driving failure probabilities. Leveraging sentiment analysis allowed the models to capture a richer, more multifaceted representation of the textual data. This study developed a novel CSL approach that integrates domain knowledge regarding the varying cost impacts of misclassifying different failure types into ML models. This research demonstrated an effective fusion of text insights from inspection reports with structured pipeline data that enhances model interpretability. The resulting AI modeling framework generated data-driven predictions of the causes of failure that could enable transportation agencies with actionable insights. These insights enable tailored preventative maintenance decisions to proactively mitigate emerging pipeline failures.

Classification of high-dimensional imbalanced biomedical data based on spectral clustering SMOTE and marine predators algorithm

The research of biomedical data is crucial for disease diagnosis, health management, and medicine development. However, biomedical data are usually characterized by high dimensionality and class imbalance, which increase computational cost and affect the classification performance of minority class, making accurate classification difficult. In this paper, we propose a biomedical data classification method based on feature selection and data resampling. First, use the minimal-redundancy maximal-relevance (mRMR) method to select biomedical data features, reduce the feature dimension, reduce the computational cost, and improve the generalization ability; then, a new SMOTE oversampling method (Spectral-SMOTE) is proposed, which solves the noise sensitivity problem of SMOTE by an improved spectral clustering method; finally, the marine predators algorithm is improved using piecewise linear chaotic maps and random opposition-based learning strategy to improve the algorithm’s optimization seeking ability and convergence speed, and the key parameters of the spectral-SMOTE are optimized using the improved marine predators algorithm, which effectively improves the performance of the over-sampling approach. In this paper, five real biomedical datasets are selected to test and evaluate the proposed method using four classifiers, and three evaluation metrics are used to compare with seven data resampling methods. The experimental results show that the method effectively improves the classification performance of biomedical data. Statistical test results also show that the proposed PRMPA-Spectral-SMOTE method outperforms other data resampling methods.

Incremental value of automatically segmented perirenal adipose tissue for pathological grading of clear cell renal cell carcinoma: a multicenter cohort study.

Accurate preoperative prediction of the pathological grade of clear cell renal cell carcinoma (ccRCC) is crucial for optimal treatment planning and patient outcomes. This study aims to develop and validate a deep-learning (DL) algorithm to automatically segment renal tumours, kidneys, and perirenal adipose tissue (PRAT) from computed tomography (CT) images and extract radiomics features to predict the pathological grade of ccRCC. In this cross-ethnic retrospective study, a total of 614 patients were divided into a training set (383 patients from the local hospital), an internal validation set (88 patients from the local hospital), and an external validation set (143 patients from the public dataset). A two-dimensional TransUNet-based DL model combined with the train-while-annotation method was trained for automatic volumetric segmentation of renal tumours, kidneys, and visceral adipose tissue (VAT) on images from two groups of datasets. PRAT was extracted using a dilation algorithm by calculating voxels of VAT surrounding the kidneys. Radiomics features were subsequently extracted from three regions of interest of CT images, adopting multiple filtering strategies. The least absolute shrinkage and selection operator (LASSO) regression was used for feature selection, and the support vector machine (SVM) for developing the pathological grading model. Ensemble learning was used for imbalanced data classification. Performance evaluation included the Dice coefficient for segmentation and metrics such as accuracy and area under curve (AUC) for classification. The WHO/International Society of Urological Pathology (ISUP) grading models were finally interpreted and visualized using the SHapley Additive exPlanations (SHAP) method. For automatic segmentation, the mean Dice coefficient achieved 0.836 for renal tumours and 0.967 for VAT on the internal validation dataset. For WHO/ISUP grading, a model built with features of PRAT achieved a moderate AUC of 0.711 (95% CI, 0.604-0.802) in the internal validation set, coupled with a sensitivity of 0.400 and a specificity of 0.781. While model built with combination features of the renal tumour, kidney, and PRAT showed an AUC of 0.814 (95% CI, 0.717-0.889) in the internal validation set, with a sensitivity of 0.800 and a specificity of 0.753, significantly higher than the model built with features solely from tumour lesion (0.760; 95% CI, 0.657-0.845), with a sensitivity of 0.533 and a specificity of 0.767. Automated segmentation of kidneys and visceral adipose tissue (VAT) through TransUNet combined with a conventional image morphology processing algorithm offers a standardized approach to extract PRAT with high reproducibility. The radiomics features of PRAT and tumour lesions, along with machine learning, accurately predict the pathological grade of ccRCC and reveal the incremental significance of PRAT in this prediction.

LSTM-CNN Hybrid Model Performance Improvement with BioWordVec for Biomedical Report Big Data Classification

The rise in mortality rates due to leukemia has fueled the swift expansion of publications concerning the disease. The increase in publications has dramatically affected the enhancement of biomedical literature, further complicating the manual extraction of pertinent material on leukemia. Text classification is an approach used to retrieve pertinent and top-notch information from the biomedical literature. This research suggests employing an LSTM-CNN hybrid model to tackle imbalanced data classification in a dataset of PubMed abstracts centred on leukemia. Random Undersampling and Random Oversampling techniques are merged to tackle the data imbalance problem. The classification model’s performance is improved by utilizing a pre trained word embedding created explicitly for the biomedical domain, BioWordVec. Model evaluation indicates that hybrid resampling techniques with domain-specific pre-trained word embeddings can enhance model performance in classification tasks, achieving accuracy, precision, recall, and f1-score of 99.55%, 99%, 100%, and 99%, respectively. The results suggest that this research could be an alternative technique to help obtain information about leukemia.

Enhancing intrusion detection with imbalanced data classification and feature selection in machine learning algorithms

Enhancing intrusion detection with imbalanced data classification and feature selection in machine learning algorithms

Kreĭn twin support vector machines for imbalanced data classification

Conventional classification assumes a balanced sample distribution among classes. However, such a premise leads to biased performance over the majority class (with the highest number of instances). The Twin Support Vector Machines (TWSVM) obtained great prominence due to their low computational burden compared to the standard SVM. Besides, traditional machine learning seeks methods whose solution depends on a convex problem or semi-positive definite similarity matrices. Yet, this kind of matrix cannot adequately represent many real-world applications. The above defines the need to use non-negative measures as an indefinite function in a Reproducing Kernel Kreĭn Space (RKKS). This paper proposes a novel approach called Kreĭn Twin Support Vector Machines (KTSVM), which appropriately incorporates indefinite kernels within a TWSVM-based gradient optimization. To code pertinent input patterns within an imbalanced data discrimination, our KTSVM employs an implicit mapping to a RKKS. Also, our approach takes advantage of the TWSVM scheme’s benefits by creating two parallel hyperplanes. This promotes the KTSVM optimization in a gradient-descent framework. Results obtained on synthetic and real-world datasets demonstrate that our solution performs better in terms of imbalanced data classification than state-of-the-art techniques.

PFSC: Parameter-free sphere classifier for imbalanced data classification

Imbalanced data classification is a prevalent challenge in real-world applications. While a conventional sphere-based classification algorithm, random sphere cover (RSC), evenly constructs a set of spheres for two classes in balanced data using a parameter for the minimum sphere size, it struggles with constructing minority spheres in class-imbalanced data. Although RSC can be combined with existing oversampling methods, this approach requires additional hyperparameters, and its effectiveness decreases as the minority size decreases. To overcome these issues, we propose a novel approach that employs the area under the receiver operating characteristic curve (AUC) to construct and expand spheres for minority class. This parameter-free sphere classifier considers both the majority and minority classes simultaneously. We conducted a thorough experiment on both synthetic and 50 real datasets, which revealed that our proposed method outperformed existing various oversampling techniques with the lowest training time.

Robust Visual Recognition with Class-Imbalanced Open-World Noisy Data

Learning from open-world noisy data, where both closed-set and open-set noise co-exist in the dataset, is a realistic but underexplored setting. Only recently, several efforts have been initialized to tackle this problem. However, these works assume the classes are balanced when dealing with open-world noisy data. This assumption often violates the nature of real-world large-scale datasets, where the label distributions are generally long-tailed, i.e. class-imbalanced. In this paper, we study the problem of robust visual recognition with class-imbalanced open-world noisy data. We propose a probabilistic graphical model-based approach: iMRF to achieve label noise correction that is robust to class imbalance via an efficient iterative inference of a Markov Random Field (MRF) in each training mini-batch. Furthermore, we design an agreement-based thresholding strategy to adaptively collect clean samples from all classes that includes corrected closed-set noisy samples while rejecting open-set noisy samples. We also introduce a noise-aware balanced cross-entropy loss to explicitly eliminate the bias caused by class-imbalanced data. Extensive experiments on several benchmark datasets including synthetic and real-world noisy datasets demonstrate the superior performance robustness of our method over existing methods. Our code is available at https://github.com/Na-Z/LIOND.

Optimizing deep learning-based intrusion detection in cloud computing environment with chaotic tunicate swarm algorithm

Intrusion Detection (ID) in cloud environments is vital to maintain the safety and integrity of data and resources. However, the presence of class imbalance, where normal samples significantly outweigh intrusive instances, poses a challenge in constructing a potential ID system. Deep Learning (DL) methods, with their capability to automatically study complex patterns and features, present a promising solution in various ID tasks. Such methods can automatically learn intricate features and patterns from the input dataset, making them suitable for detecting anomalies and finding intrusions in cloud environments. Therefore, this study proposes a Class Imbalance Data Handling with an Optimal Deep Learning-Based Intrusion Detection System (CIDH-ODLIDS) in a cloud computing atmosphere. The CIDH-ODLIDS technique leverages optimal DL-based classification and addresses class imbalance. Primarily, the CIDH-ODLIDS technique preprocesses the input data using a Z-score normalization approach to ensure data quality and consistency. To handle class imbalance, the CIDH-ODLIDS technique employs oversampling techniques, particularly focused on synthetic minority oversampling techniques such as Adaptive Synthetic (ADASYN) sampling. ADASYN generates synthetic instances for the minority class depending on the available data instances, effectively balancing the class distribution and mitigating the impact of class imbalance. For the ID process, the CIDH-ODLIDS technique utilizes a Fuzzy Deep Neural Network (FDNN) model, and its tuning procedure is performed using the Chaotic Tunicate Swarm Algorithm (CTSA). CTSA is employed to choose the learning rate of the FDNN methods optimally. The experimental assessment of the CIDH-ODLIDS method is extensively conducted on three IDS datasets. The comprehensive comparison results confirm the superiority of the CIDH-ODLIDS algorithm over existing techniques.

Clustering-based incremental learning for imbalanced data classification

Imbalanced data classification presents a significant challenge when there is a substantial disparity in sample sizes across different classes. This issue severely affects classifier accuracy in predicting minority classes, hampering numerous real-world applications. Traditional methods address data imbalance by using undersampling or oversampling techniques. However, these methods may lead to information loss during sample reduction or introduce noise and model bias through synthetic sample generation. In this paper, we introduce DRIL, an innovative clustering-based incremental learning approach designed to overcome these limitations and improve the classification of minority class samples. Specifically, we employ a “two-step clustering” method to rebalance the dataset, partitioning it into similar and representative sub-dataset. Subsequently, incremental learning is applied to enable the classifier to gradually acquire knowledge about these sub-data, establishing a comprehensive understanding of all features present in the imbalanced dataset. Experimental results on twenty datasets demonstrate that our incremental learning-based algorithm outperforms baseline methods in correctly classifying minority classes while exhibiting improved precision and F1 score performance.

Deep convolutional neural networks with genetic algorithm-based synthetic minority over-sampling technique for improved imbalanced data classification

Imbalanced data classification presents a challenge in machine learning, inducing biased model learning. Moreover, data dimensionality poses another challenge as it highly impacts classifier performance. This paper proposes a new deep-learning method that combines feature selection with oversampling to address these challenges. The proposed approach, GA-SMOTE-DCNN, integrates a genetic algorithm (GA) for feature selection, SMOTE for oversampling, and a deep 1D-convolutional neural network (DCNN) for classification. This study reveals that pre-splitting the data into training and testing sets before applying SMOTE results in higher accuracy, showing an improvement in accuracy ranging between 1.94% and 3.98% compared to post-SMOTE splitting for each dataset. This method achieved accuracy rates of 86.81% for the Balance Scale dataset, 86.15% for the Oil Spill dataset, 89.21% for the Yeast dataset, 91.32% for the Mammography dataset, 88.23% for the Australian credit dataset, and 89.53% for the German Credit dataset when compared with benchmark methods, underscoring its significance in tackling high-dimensional and imbalanced data classification problems. This method demonstrates scalability in effectively addressing challenges associated with high-dimensional and imbalanced data classification across various domains.

A Synthetic Minority Oversampling Technique Based on Gaussian Mixture Model Filtering for Imbalanced Data Classification.

Data imbalance is a common phenomenon in machine learning. In the imbalanced data classification, minority samples are far less than majority samples, which makes it difficult for minority to be effectively learned by classifiers. A synthetic minority oversampling technique (SMOTE) improves the sensitivity of classifiers to minority by synthesizing minority samples without repetition. However, the process of synthesizing new samples in the SMOTE algorithm may lead to problems such as "noisy samples" and "boundary samples." Based on the above description, we propose a synthetic minority oversampling technique based on Gaussian mixture model filtering (GMF-SMOTE). GMF-SMOTE uses the expected maximum algorithm based on the Gaussian mixture model to group the imbalanced data. Then, the expected maximum filtering algorithm is used to filter out the "noisy samples" and "boundary samples" in the subclasses after grouping. Finally, to synthesize majority and minority samples, we design two dynamic oversampling ratios. Experimental results show that the GMF-SMOTE performs better than the traditional oversampling algorithms on 20 UCI datasets. The population averages of sensitivity and specificity indexes of random forest (RF) on the UCI datasets synthesized by GMF-SMOTE are 97.49% and 97.02%, respectively. In addition, we also record the G-mean and MCC indexes of the RF, which are 97.32% and 94.80%, respectively, significantly better than the traditional oversampling algorithms. More importantly, the two statistical tests show that GMF-SMOTE is significantly better than the traditional oversampling algorithms.

MVQS: Robust multi-view instance-level cost-sensitive learning method for imbalanced data classification

Multi-view imbalanced learning is to handle the datasets with multi-view representations and imbalanced classes. Existing multi-view imbalanced learning methods can be divided into two main categories: multi-view ensemble learning and multi-view cost-sensitive learning. However, these methods suffer from the following problems: 1) neglecting either consensus or complementary information, 2) complex preprocessing and information fusion in multi-view ensemble learning and manual assignment of misclassification costs in multi-view cost-sensitive learning, and 3) limited ability to handle noisy samples. Therefore, we aim to design a concise and unified framework to grapple with the multi-view representations, imbalanced classes and noisy samples simultaneously. Inspired by the merits of support vector machine (SVM) and quadratic type squared error (QTSE) loss function, we propose a robust multi-view instance-level cost-sensitive SVM with QTSE loss (MVQS) for imbalanced data classification. The consensus regularization term and combination weight strategy are employed to fully exploit multi-view information. The QTSE loss can adaptively impose instance-level penalties to the misclassification of samples, and make MVQS be robust to noisy samples. We solve MVQS with the alternating direction method of multipliers (ADMM) and the gradient descent (GD) algorithm. Comprehensive experiments validate that MVQS is more competitive and robust than other benchmark approaches.

Regularized Discrete Optimal Transport for Class-Imbalanced Classifications

Imbalanced class data are commonly observed in pattern analysis, machine learning, and various real-world applications. Conventional approaches often resort to resampling techniques in order to address the imbalance, which inevitably alter the original data distribution. This paper proposes a novel classification method that leverages optimal transport for handling imbalanced data. Specifically, we establish a transport plan between training and testing data without modifying the original data distribution, drawing upon the principles of optimal transport theory. Additionally, we introduce a non-convex interclass regularization term to establish connections between testing samples and training samples with the same class labels. This regularization term forms the basis of a regularized discrete optimal transport model, which is employed to address imbalanced classification scenarios. Subsequently, in line with the concept of maximum minimization, a maximum minimization algorithm is introduced for regularized discrete optimal transport. Subsequent experiments on 17 Keel datasets with varying levels of imbalance demonstrate the superior performance of the proposed approach compared to 11 other widely used techniques for class-imbalanced classification. Additionally, the application of the proposed approach to water quality evaluation confirms its effectiveness.

Class imbalance data handling with optimal deep learning-based intrusion detection in IoT environment

Class imbalance data handling with optimal deep learning-based intrusion detection in IoT environment